Search for: All records

For any given neural network architecture a permutation of weights and biases results in the same functional network. This implies that optimization algorithms used to 'train' or 'learn' the network are faced with a very large number (in the millions even for small networks) of equivalent optimal solutions in the parameter space. To the best of our knowledge, this observation is absent in the literature. In order to narrow down the parameter search space, a novel technique is introduced in order to fix the bias vector configurations to be monotonically increasing. This is achieved by augmenting a typical learning problem with inequality constraints on the bias vectors in each layer. A Moreau-Yosida regularization based algorithm is proposed to handle these inequality constraints and a theoretical convergence of this algorithm is established. Applications of the proposed approach to standard trigonometric functions and more challenging stiff ordinary differential equations arising in chemically reacting flows clearly illustrate the benefits of the proposed approach. Further application of the approach on the MNIST dataset within TensorFlow, illustrate that the presented approach can be incorporated in any of the existing machine learning libraries.

 

Abstract We consider the problem of waves propagating in a viscoelastic solid. For the material properties of the solid we consider both classical and fractional differentiation in time versions of the Zener, Maxwell, and Voigt models, where the coupling of different models within the same solid are covered as well. Stability of each model is investigated in the Laplace domain, and these are then translated to time-domain estimates. With the use of semigroup theory, some time-domain results are also given which avoid using the Laplace transform and give sharper estimates. We take the time to develop and explain the theory necessary to understand the relation between the equations we solve in the Laplace domain and those in the time-domain which are written using the language of causal tempered distributions. Finally we offer some numerical experiments that highlight some of the differences between the models and how different parameters effect the results. 

We consider an optimal control problem where the state equations are a coupled hyperbolic–elliptic system. This system arises in elastodynamics with piezoelectric effects—the elastic stress tensor is a function of elastic displacement and electric potential. The electric flux acts as the control variable and bound constraints on the control are considered. We develop a complete analysis for the state equations and the control problem. The requisite regularity on the control, to show the well-posedness of the state equations, is enforced using the cost functional. We rigorously derive the first-order necessary and sufficient conditions using adjoint equations and further study their well-posedness. For spatially discrete (time-continuous) problems, we show the convergence of our numerical scheme. Three-dimensional numerical experiments are provided showing convergence properties of a fully discrete method and the practical applicability of our approach.